focus retriev evalu tradit document retriev shown competit approach xml element retriev counter intuit element retriev task request relev document part retriev paper conduct compar analysi document element retriev highlight rel strength weak approach explain rel effect document retriev approach element retriev task workshop exploit semant annot inform retriev esair cikm workshop increas amount structur web result modern web languag user tag annot emerg robust nlp tool meaning semant annot hold promis significantli enhanc inform access enhanc depth analysi todai system start explor possibl begin understand valuabl semant cue put fruit unleash potenti semant annot requir box combin insight natur languag process nlp bag word insight databas db structur effici aggreg million record insight inform retriev ir effect goal direct search evalu insight knowledg manag km grip greater workshop aim bring research disciplin work greatest challeng year desir result workshop concret insight potenti semant annot concret step research forward synchron relat research happen nlp db ir km wai combin strength disciplin live interact workshop contribut inspir attende box entiti rank wikipedia pivot paper investig task entiti rank web searcher entiti arguabl serv present rank list entiti directli list web page relev potenti redund inform entiti entiti repres web homepag naiv approach entiti rank standard text retriev experiment result demonstr text retriev effect find relev page perform poorli find entiti propos wikipedia pivot find entiti web allow reduc hard web entiti rank problem easier problem wikipedia entiti rank wikipedia properli identifi entiti characterist wikipedia elabor categori structur handl entiti type main search log dbase approach evalu abstract offer content digit librari natur interest assess perform system meet user inform standard evalu benchmark develop inform retriev test retriev effect gener benchmark focu singl document genr languag media type searcher stereotyp radic uniqu content user commun digit librari paper propos deriv domain specif test collect readili interact data search log file captur domain specif digit librari archiv institut complet search log span multipl year deriv larg scale test collect result domain specif test collect evalu rang retriev model final manual deriv set topic judg human expert base set mail refer question respons archivist valid main find deriv massiv reliabl domain specif test collect search log file languag model andword cloud empir studi queri specif abstract analys statist behavior queri quantiti queri log sum idf queri term queri specif queri specif narrow possibl model distribut gamma log normal log logist depend queri length sum consid result applic queri perform predict artifici queri gener import anchor text ad hoc search revisit impact collect size relev divers impact collect size relev divers link wikipedia web contact qualiti higher educ multi dimension concept li effect transmit knowledg skill authent content coverag depth inform avail read teach materi remov obstacl learn applic knowledg solv real life problem fruit knowledg person social domain converg content varieti knowledg space countri region section peopl cost effect administr effici inform technolog progress fast decad produc equip afford cost made wider applic feasibl technolog made search gather dissemin store retriev transmiss recept knowledg easier cheaper faster side side vast virtual librari vy librari print emerg continu grow rapidli hold elibrari librari tomorrow librari print antiqu archiv object past paper discuss detail inform technolog appli enhanc qualiti higher educ afford cost discuss major obstacl optim util inform technolog measur remov link wikipedia web contact qualiti higher educ multi dimension concept li effect transmit knowledg skill authent content coverag depth inform avail read teach materi remov obstacl learn applic knowledg solv real life problem fruit knowledg person social domain converg content varieti knowledg space countri region section peopl cost effect administr effici inform technolog progress fast decad produc equip afford cost made wider applic feasibl technolog made search gather dissemin store retriev transmiss recept knowledg easier cheaper faster side side vast virtual librari vy librari print emerg continu grow rapidli hold elibrari librari tomorrow librari print antiqu archiv object past paper discuss detail inform technolog appli enhanc qualiti higher educ afford cost discuss major obstacl optim util inform technolog measur remov current research focus retriev result aggreg abstract focus retriev result aggreg provid user answer inform pointer document focus retriev identifi relev document part document relev reduc time take user navig document result aggreg fulfil user draw inform sourc collect document part document aggreg singl result reduc time take user fulfil inform special issu includ paper show breadth depth current state research branch inform retriev includ descript live result aggreg system experiment focuss retriev system includ sentenc retriev question answer entiti rank metric passag retriev introduc special issu provid overview work present paper current research focus retriev result aggreg abstract focus retriev result aggreg provid user answer inform pointer document focus retriev identifi relev document part document relev reduc time take user navig document result aggreg fulfil user draw inform sourc collect document part document aggreg singl result reduc time take user fulfil inform special issu includ paper show breadth depth current state research branch inform retriev includ descript live result aggreg system experiment focuss retriev system includ sentenc retriev question answer entiti rank metric passag retriev introduc special issu provid overview work present paper report inex inex investig focus retriev structur document provid larg test collect structur document uniform evalu measur forum organ compar result paper report inex evalu campaign consist wide rang track ad hoc book effici entiti rank interact qa link wiki xml mine inex run volunt effort ir research commun idea time spend major impact advanc focus retriev signal dto dnois approach score normal score normal indispens distribut retriev fusion meta search merg result list requir distribut approach score normal refer relev binari mixtur model normal exponenti suffer lack univers troublesom paramet estim spars relev develop approach tackl problem aggreg score distribut refer relev suitabl uncoop engin method base assumpt score produc engin consist signal nois compon approxim submit defin set artifici queri engin evalu standard distribut retriev testb show signal nois approach yield result distribut method signific product investig queri length distribut normal frequenc evalu context search archiv find aid retriev origin order abstract archiv principl proven keep materi creator corollari origin order keep order creation intact improv access archiv materi investig import relev rank origin order search find aid ead xml retriev experi show relev rank paramount import origin order retriev result tend cluster origin order score distribut inform retriev abstract review histori model score distribut focus mixtur normal exponenti investig theoret empir evid support discuss previous suggest condit valid binari mixtur model satisfi recal fallout convex hypothesi formul hypothes compon distribut limit condit paramet valu mixtur suggest past current theoret argument point gamma univers model normal exponenti usabl approxim theoret contribut provid experiment evid show vector space geometr model bm friendli normal exponenti convex problem mixtur possess practic sever link document import topic relev abstract web inform retriev web link structur sourc evid global link evid natur queri independ direct indic topic relev document search request result link inform consid identifi import document local link evid contrast queri depend principl relat topic relev analys link evid wikipedia larg set ad hoc retriev topic relev judgement investig relat link evid topic relev analysi inex ad hoc track result focus search book wikipedia categori link relev feedback overview inex ad hoc track abstract paper overview inex ad hoc track main goal ad hoc track fold goal investig impact collect scale markup collect base wikipedia time larger longer articl addit semant annot reason ad hoc track task stai unchang task inex return goal studi impact verbos queri retriev effect markup structur constraint wikipedia layout base markup enrich semant markup phrase goal compar result granular allow system retriev xml element rang xml element arbitrari passag text investig intern document structur provid xml mark retriev relev inform inex ad hoc stop read rank list threshold optim truncat score distribut rank retriev disadvantag comparison tradit boolean retriev clear cut point stop consult result problem setup investig develop method select rank cutoff optim effect measur assum input system output queri document score distribut task essenti score distribut threshold optim problem recent trend model score distribut normal exponenti mixtur normal relev exponenti relev document score discuss main theoret problem current model support incompat convex develop model address main contribut paper truncat normal exponenti model vari outtrunc score rang handl conduct rang experi captur structur debat transcipt meet document genr character complex narr structur essenc paper investig semant annot speaker order captur debat structur relat content debat structur visual graph content condens word cloud creat parsimoni languag model evalu show tool adequ captur structur content debat aggreg level categori subject descriptor focus search digit archiv abstract present system descript archiv inform system approach gain onlin access digit archiv creat metadata standard encod archiv descript ead show aggreg base system develop archiv data xml inform retriev xml ir describ stage compon index digit find aid xml databas subsequ queri retriev inform databas eventu deliveri inform user contextu interfac wikipedia link structur paper investig differ wikipedia web link structur respect indic relev page topic request experiment evid ir test collect gov collect trec web track wikipedia xml corpu inex perform compar analysi wikipedia gov link structur investig link evid improv search wikipedia gov domain main find wikipedia link structur similar web dens link wikipedia outlink behav similar inlink good indic relev web inlink import incorpor link evid retriev model wikipedia global link evid fail local context account index unit cowl foundat discuss paper report inex inex investig focus retriev structur document provid larg test collect structur document uniform evalu measur forum organ compar result paper report inex evalu campaign consist wide rang track ad hoc book effici entiti rank interact qa link wiki xml mine focus access xml document report sigir workshop futur ir evalu focus access xml document access archiv find aid context matter abstract detail design search engin archiv find aid base xml databas system result system show result vari granular individu archiv item fond context archiv present preserv archiv structur provid import contextu inform individu result click warp user full find aid select part focu import link evid wikipedia abstract wikipedia popular inform sourc web free encyclopedia dens link link structur wikipedia differ web larg intern link wikipedia typic base word natur occur page link semant relat entri main aim find wikipedia link structur exploit improv ad hoc inform retriev analys relat wikipedia link relev page experi link evid focus retriev wikipedia content base test collect inex main find analysi link structur reveal wikipedia link structur possibl weak indic relev experi inex ad hoc retriev task reveal link evid made sensit local context signific improv retriev effect contrast earlier trec experi crawl web data shown wikipedia link structur improv effect ad hoc retriev access archiv materi context archiv find aid long complexli structur document describ archiv materi paper trail live corpor bodi person famili find aid encod xml standard encod archiv descript ead made public web site archiv institut provid access long complexli structur document hand user tend specif archiv materi deepli nest insid archiv hand interpret mean item crucial depend context insight field xml retriev subfield inform retriev recent attract lot attent annual evalu effort inex develop system search collect digit find aid fundament choic archiv access system access fond archiv system direct access individu archiv materi level descript system retriev archiv materi preserv origin context paper report result extens user studi system main find test person prefer system retriev archiv materi origin context test person indic system assist assess relev navig direct access relev part find aid overview inex ad hoc track abstract paper overview inex ad hoc track main goal ad hoc track fold goal investig intern document structur provid xml mark retriev relev inform continu inex reason retriev result liber arbitrari passag measur chosen fairli compar system retriev element rang element arbitrari passag goal compar focus retriev articl retriev directli earlier year reason standard document retriev rank deriv run evalu standard measur addit set queri target wikipedia deriv proxi log run evalu click wikipedia page inex ad hoc track featur task focus task rank list nonoverlap result element passag need relev context task overlap result element passag return group articl context task singl start point element start tag passag start articl need discuss result task examin rel effect element passag retriev examin context content keyword search content structur ca structur search final abil focus retriev techniqu rank articl standard document retriev techniqu judg topic queri click proxi log impact document level rank focus retriev abstract document retriev techniqu proven competit method evalu focus retriev focus approach xml element retriev passag retriev locat relev text document larger context document lead superior document level rank paper investig impact document retriev rank collect inex ad hoc book track short document wikipedia collect longer book book track collect experi method combin document element retriev approach find world improv individu retriev strategi retain document rank document retriev approach replac document retriev element element retriev approach document level rank posit impact focus retriev wikipedia impact longer book book track collect find entiti wikipedia link categori content dbase link detect approach vector space model abstract link detect special applic focus retriev paper present content base link detect approach vector space model present result conclud discuss merit defici approach link classifi wikipedia page abstract paper descript experi inex xml mine track goal xml mine track explor link inform improv classif accuraci approach propag categori probabl link page find link inform lead margin improv baselin naiv bay model initi misclassifi page link inform nois parsimoni languag model web data paper explor parsimoni languag model web retriev model smaller effici standard languag model suit larg scale web retriev conduct experi trec topic set found parsimoni languag model result improv retriev effect standard languag model data set measur case improv signific substanti earlier experi newspap newswir data studi queri length analys queri length fit power law poisson distribut queri set provid practic model queri length base truncat poisson distribut short queri power law distribut longer queri fit real queri length distribut earlier propos locat relev text xml document tradit document retriev shown competit approach xml element retriev counter intuit element retriev task request relev document part retriev paper conduct compar analysi document element retriev highlight rel strength weak approach explain rel effect document retriev approach element retriev task impact posit neg topic relev feedback abstract document descript experi relev feedback track experi amount feedback includ neg relev feedback feedback implement massiv weight queri expans parsimoni queri expans relev document jelinek mercer smooth perform relev feedback track dataset addit blind feedback result blind feedback set size explicit feedback set small number topic topic feedback appli turn benefici earli precis stop read rank list abstract document particip trec legal track year focus sole select rank cut off optim evalu measur topic report sigir workshop focus retriev singapor paper present session fourth session joint overview inex ad hoc track abstract paper overview inex ad hoc track main purpos ad hoc track investig intern document structur provid xml markup retriev relev inform reason retriev result liber arbitrari passag measur chosen fairli compar system retriev element rang element arbitrari passag inex ad hoc track featur task focus task rank list overlap result element passag need relev context task overlap result element passag return group articl context task singl start point element start tag passag start articl need discuss result task examin rel effect element passag retriev examin context content keyword search content structur ca structur search inex evalu measur abstract paper describ offici measur retriev effect emploi ad hoc track inex earlier year xml element retriev result format liber arbitrari passag respons inex measur base amount highlight text retriev lead natur extens establish measur precis recal measur defin focus task evalu interpol precis recal ip term highlight text retriev relev context task evalu averag gener precis magp whereth gener score articl base retriev highlight text context task evalu averag gener precis magp gener score articl base distanc assessor entri point unifi access heterogen data cultur heritag paper address prototyp problem cultur heritag institut ambit disclos content singl unifi system enterpris institut heterogen collect distribut multipl legaci system approach turn metadata retriev problem free text retriev problem uncondit merg heterogen collect flatten metadata structur investig viabil approach extens case studi larg museum main find convert digit content text index standard ir system effect build unifi system provid access data initi empir evalu show superior perform comparison legaci system institut find approach viabl option give access heterogen collect detect link wikipedia abstract paper document effort inex particip ad hoc track link wiki track interact track continu inex main aim inex ad hoc track investig effect incorpor link evid model ca filter method exploit structur hint inex topic link wiki track investig rel effect link detect base retriev similar document vector space model filter name wikipedia articl establish link interact track part interact experi compar element retriev system passag retriev system main result ad hoc track link prior improv run relev context context task ca pool filter effect relev context context task link wiki track result show detect link match work link gener gener hurt perform interact track test person show weak prefer element retriev system passag retriev system unifi access heterogen data cultur heritag paper address prototyp problem cultur heritag institut ambit disclos content singl unifi system enterpris institut heterogen collect distribut multipl legaci system approach turn metadata retriev problem free text retriev problem uncondit merg heterogen collect flatten metadata structur investig viabil approach extens case studi larg museum main find convert digit content text index standard ir system effect build unifi system provid access data initi empir evalu show superior perform comparison legaci system institut find approach viabl option give access heterogen collect start read textual xml document structur inform retriev aim exploit document structur retriev relev compon allow user straight relev materi paper call entri point bep intend give user start point access relev inform document examin relationship bep relev compon inex ad hoc assess main find document short assessor choos entri point distanc start document entri point coincid relev charact relev document show strong relat bep relev text find brows bep articl singl relev passag contain bep context bep articl relev passag evalu relev context document retriev twist access legal document exact match match combin abstract paper document effort particip trec legal track multipl aim experi queri formul exploit verbos topic statement analys rank retriev method fruitfulli combin tradit boolean queri main find summar mix result combin origin search request term extract verbos topic statement combin boolean refer run wit rank retriev run high recal boolean retriev whilst precis score show improv boolean rank retriev run found treat boolean queri free text vari degre interpret origin oper competit result type queri captur relev document combin request text boolean queri lead substanti gain precis recal parsimoni languag model terabyt text report sigir workshop focus retriev report overview webclef abstract report clef webclef track devot crosslingu web retriev provid detail retriev task topic set result particip webclef stream item topic consist manual topic includ select webclef topic set topic ii automat gener topic gener techniqu result topic show current clir system effect retriev averag target page top rank manual construct topic result higher perform automat gener final result score automat topic provid reason rank system show automat gener topic attract altern situat manual topic readili cross dlanguag approach histor document retriev introduct cultur heritag preserv librari archiv museum made document written centuri ago larg scale digit initi digicult make document expert user digit librari vertic search engin user queri histor document collect disappoint experi natur languag evolv time chang pronunci spell word introduc continu older word inex evalu measur abstract paper describ offici measur retriev effect emploi ad hoc track inex cross dlanguag approach histor document retriev introduct cultur heritag preserv librari archiv museum made document written centuri ago larg scale digit initi digicult make document expert user digit librari vertic search engin user queri histor document collect disappoint experi natur languag evolv time chang pronunci spell word introduc continu older word filter cluster xml retriev result abstract part inex adhoc track conduct rang experi filter cluster xml element retriev result basic retriev engin retriev arbitrari element collect task run filter remov textual overlap element focus task result run cluster articl context task final select element articl context task main find complet element index outperform restrict index base section structur albeit differ small group overlap element articl lead perform degrad improv score restrict pure element run remov overlap group element articl select singl element articl lead moder loss precis inex evalu measur abstract paper describ offici measur retriev effect emploi ad hoc track inex experi document queri represent terabyt text track conduct rang experi investig effect larger test collect adhoc item topic look amount smooth requir larg scale collect found largescal collect requir smooth investig rel effect web centric document represent base document text incom anchor text page titl found adhoc task provid crucial addit retriev cue name page find task studi rel effect queri represent short verbos statement topic request intermedi queri base characterist term topic statement found verbos queri lead improv retriev effect articul inform xml queri languag document centric xml mixtur text structur increas avail document centric xml document queri facil structur constraint constraint content document express express languag queri xml document user express inform address question experiment theoret point view experiment analysi compar structur ignor structur awar retriev approach test suit inex xml retriev evalu initi theoret creat mathemat model user knowledg set document defin queri languag fit model languag correspond xml version field search inex queri languag main experiment find structur vari degre complex third queri express field search format hierarch structur document quarter queri constraint context element return contextu constraint captur ordinari keyword queri structur search hint strict requir judg underli inform fourth structur queri function precis enhanc devic web dcentric languag model investig languag model inform navig web search retriev web task differ substanti ordinari ad hoc retriev perform analysi prior probabl relev wide rang content featur shed light import content featur web retriev languag model natur incorpor multipl document represent content inform emploi mixtur languag model base document full text incom anchor text document titl studi rang prior base document length url structur link topolog type topic distil home page name page mix queri set find mixtur model lead consider improv retriev effect topic type web centric prior gener lead improv retriev effect structur queri xml retriev mark xml tag tag add lightweight semant text queri collect call hybrid queri languag text rich natur document suggest content orient ir approach mark user add structur constraint ir queri propos approach hybrid contentand structur queri decompos queri multipl content queri result combin wai determin structur constraint origin queri report ongo work present preliminari evalu result base inex test set overview webclef eurogov engin multilingu web corpu abstract eurogov multilingu web corpu creat serv document collect webclef clef web retriev task eurogov collect web page crawl european union portal european union member state government web site russian government web site corpu million document written european languag paper provid detail descript eurogov collect combin method crosslingu web retriev abstract investig rang crosslingu web retriev task test suit clef webclef track featur stream item topic languag main find straightforward index retriev effect mix monolingu web retriev ii standard machin translat method effect bilingu web retriev iii standard combin method ineffect multilingu web retriev analyz failur suggest altern score normal lead effect multilingu retriev result effect structur queri select index xml retriev inex ad hoc track cover focus fetch brows task structur counterpart research question round inex threefold main research question investig contribut structur constraint improv retriev perform main result type structur constraint effect constrain target result element improv term earli precis constrain context result element improv averag precis research question experi select index strategi base length element tag element consid relev earlier inex year simpli index section articl experi show disregard total number element decreas retriev perform consid automat creation structur queri blind feedback result inconclus due queri lack comparison tradit blind feedback user xml element retriev system inex interact track focus compar experi baselin system daffodil hyrex compar home grown xml element retriev system xmlfind xmlfind system interfac xml inform retriev search engin index individu xml element ieee collect main find test person show appreci system xmlfind receiv higher score daffodil interfac structur depend retriev element account retriev element overlap part test person regard problemat gener opinion test person us xml retriev system unequivoc posit respons highlight hope advantag xml retriev system biomed retriev thesauru abstract search special collect biomed literatur typic requir intim knowledg special terminolog disappoint experi know term unawar synonym variat terminolog result low recal score studi role thesauru biomed inform retriev process start give descript vocabulari mismatch problem natur languag queri relev document biomed literatur search provid detail case studi observ impact vocabulari mismatch problem retriev effect addition analyz mesh thesauru term index document collect base observ propos method exploit mesh thesauru improv retriev effect specif increas recal carri seri thesauru base retriev experi show substanti perform improv conclud detail analysi retriev result combin thesauri dbase method biomed retriev abstract paper describ particip effect smooth terabyt text track conduct rang experi investig effect larger collect main find summar test retriev system scale terabyt scale collect found retriev system handl million document term index time approach limit distribut retriev system hope find result earlier web track carri task item search found hand indegre url prior promot retriev effect hand combin document represent improv retriev effect investig role smooth collect size found larger collect requir smooth adhoc task smooth lead substanti gain retriev effect import length normal xml retriev process content dorient xpath queri document centric xml collect text rich document mark xml tag add lightweight semant text queri collect call hybrid queri languag text rich natur document suggest content orient ir approach mark user add structur constraint ir queri hybrid queri tend express lead principl retriev perform practic process hybrid queri ir system turn trivial delic balanc structur content inform sought propos approach process hybrid content structur queri decompos queri multipl content queri result combin wai determin structur constraint origin queri evalu method inex test suit show effect wai process content orient xpath queri trivial differ effect topic type process method retriev effect improv effect translat token combin cross dlingual retriev approach cross lingual document retriev start assumpt effect monolingu retriev core cross languag retriev system devot attent crucial ingredi approach cross lingual retriev effect token techniqu essenti cope morpholog variat common european languag effect combin method combin strategi final effect translat method translat queri document turn monolingu retriev system cross lingual retriev system proper viabil approach shown seri experi monolingu bilingu multilingu retriev improv retriev effect rerank document base control vocabulari abstract common avail classif term onlin text collect digit librari manual assign keyword kei phrase control vocabulari scientif collect goal explor addit classif inform improv retriev effect earlier research explor effect ad classif term user queri lead improv explor feedback techniqu rerank set initi retriev document base control vocabulari term assign document reli avail special dictionari thesauri comput mean control vocabulari term base occurr collect rerank strategi significantli improv retriev effect domain specif collect experiment evalu german girt french amarylli collect test suit cross languag evalu forum clef mixtur model overlap structur hint xml element retriev institut univers amsterdam complet revamp xml retriev system implement mixtur languag model top standard search engin speed structur reason index collect structur separ databas main find show blind feedback improv retriev effect increas overlap remov overlap result set decreas retriev effect metric xml cumul gain measur show ignor structur constraint good result measur term averag precis structur constraint achiev high initi precis final provid detail analysi characterist run base analysi argu explicit definit inex retriev task need effect combin inform retriev strategi european languag build effect inform retriev system requir design choic rang weight scheme type morpholog normal combin run standard techniqu reap benefit run type systemat studi effect combin strategi carri english paper exploratori overview effect combin method european languag demonstr combin effect inform retriev strategi lead signific improv retriev effect analyz rel impact retriev relev document improv rank relev document experiment evid obtain testsuit cross languag evalu forum clef multipl sourc evid xml retriev document centric xml collect text rich document mark xml tag tag add lightweight semant text queri collect call hybrid queri languag text rich natur document suggest content orient ir approach mark user add structur constraint ir queri show evid relev di erent sourc help answer hybrid queri evalu method inex test set show structur hint hybrid queri improv retriev ectiv length normal xml retriev xml retriev departur standard document retriev individu xml element rang italic word phrase full blown articl potenti retriev unit distribut xml element length unlik observ standard document collect prompt revisit issu document length normal perform compar analysi arbitrari element versu relev element show import length paramet xml retriev languag model framework investig rang techniqu deal length directli indirectli observ length bia introduc amount smooth show import extrem length prior xml retriev show simpli remov shorter element index introduc cut creat document length normal increas minim size xml element occur index import extrem length bia remain process content dand dstructur queri xml retriev mark xml tag tag add lightweight semant text queri collect call hybrid queri languag text rich natur document suggest content orient ir approach mark user add structur constraint ir queri propos approach hybrid contentand structur queri decompos queri multipl content queri result combin wai determin structur constraint origin queri report ongo work present preliminari evalu result base inex test set languag model search web corpora describ particip dmatch queri document dcentric xml web pervas xml give lightweight semant textual collect documentcentr xml collect requir queri languag gracefulli handl structur constraint constraint free text document main contribut fold outlin fragment xpath tailor user vari degre understand xml structur give syntact semant character fragment extend xpath function match semant base relev document compon express inform evalu result queri languag inex test suit show match approach outperform exact match approach evalu content structur queri monolingu document retriev european languag recent year wit consider advanc inform retriev european languag english give overview commonli techniqu analyz respect impact retriev ectiv techniqu consid rang linguist motiv techniqu morpholog normal compound split knowledg free approach gram index evalu carri data clef campaign cover european languag result show languag modicum linguist techniqu lead improv retriev ectiv languag independ techniqu languag ddepend languag dindepend approach cross dlingual text retriev investig ectiv languag depend approach document retriev stem decompound constrast languag independ approach charact gram order reap benefit type approach ectiv combin type approach focu document retriev european languag dutch english finnish french german italian russian spanish swedish di erent cross lingual inform retriev task monolingu bilingu multilingu domain specif retriev experiment evid obtain test suit cross languag evalu forum clef xml retriev retriev fundament differ standard inform retriev xml retriev unit retriev tradit ir unit retriev fix complet document xml retriev xml element document retriev unit make xml retriev difficult relev retriev unit larg small research present compar analysi approach xml retriev aim shed light xml element retriev experiment evalu data initi evalu xml retriev inex approach robust web retriev describ particip trec robust web track robust track experi impact stem feedback worst score topic main find effect stem poorli perform topic shed light role morpholog normal inform retriev home name page find topic distil task web track experi document represent retriev model main find effect anchor text index task suggest compact document represent fruit strategi scale retriev system combin evid cross dlanguag inform retriev paper describ offici run team clef part monolingu task english languag clef document collect dutch finnish french german italian spanish swedish conduct experi bilingu task english dutch english german part girt amarylli task final experi combin run situat calculu hybrid logic step abstract situat calculu origin conceiv john mc carthi main represent languag artifici intellig origin paper introduc situat calculu highlight connect field artifici intellig philosoph logic modal logic belief knowledg tens modal logic chang enorm paper set reviv connect situat calculu modal logic show quantifi hybrid logic qhl express situat calculu formula natur concis origin formul main contribut paper quantifi hybrid logic tempor oper action modal tailor made express fluent situat calculu import morpholog normal xml retriev current inform retriev system typic ignor structur aspect document sole focus textual content document addit structur form html xml sgml mark pervas internet xml retriev task present number challeng inform retriev longer reli unit retriev fix impli effect standard ir techniqu morpholog normal method carri task paper describ fulli automat run inex task submit languag infer technolog group univers amsterdam investig effect standard approach morpholog normal linguist motiv stem algorithm knowledg poor charact gram techniqu result show morpholog normal import issu xml retriev measur combin run gram run perform stem run notion indistinguish semant web languag abstract paper review notion express descript logic kurtonina de rijk express concept express order descript logic artifici intellig exemplifi develop semant web languag notion bisimul character descript logic alc direct link field sociolog call social network analysi perspect data field data repres label graph fit model intuit web languag oil daml oil exemplifi studi trophic network connect establish web languag hybrid logic extens oil limit form refer propos univers amsterdam trec describ particip trec novelti question answer web track provid detail account idea underli approach task run flexir inform retriev system situat calculu hybrid logic step abstract situat calculu origin conceiv john mc carthi main represent languag artifici intellig origin paper introduc situat calculu highlight connect field artifici intellig philosoph logic modal logic belief knowledg tens modal logic chang enorm paper set reviv connect situat calculu modal logic show quantifi hybrid logic qhl express situat calculu formula natur concis origin formul main contribut paper quantifi hybrid logic tempor oper action modal tailor made express fluent situat calculu criteria formal theori build appli logic autom reason tool social scienc paper practic operation criteria evalu scientif theori consist falsifi theori sound infer account definit precis formul criteria tailor autom theorem prover autom model gener gener tool field autom reason criteria illustr appli order logic represent classic organ theori thompson organ action formal theori build autom reason tool merit repres scientif theori formal logic express scientif theori formal logic explic theori logic formal criteria evalu theori sound consist hand criteria correspond natur question ask theori theori contradict free theori logic consist theoret argument valid theorem soundli deriv premis question hand test criteria amount make specif proof attempt model search theori model find proof theorem result test criteria quickli defi manual process fortun autom reason valuabl tool endeavor paper discuss order logic exist autom rea 